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ABSTRACT 

Recent contentions that the settings and problems of 
social evaluation research renders this new **subdiscipline^* 
substantively distinct *^rom the remainder of social psychology are 
critically examined. It is argued that the demonstrable existence of 
meta-conf licts about the conduct of social evaluation efforts, 
political sensitivities in the research arena, and the adaptive 
instability of evaluated programs do not functionally segregate 
social evaluation from traditional social psychology. Rather, these 
problems occur, hoiilbmorphically in both arenas, although they are more 
often articulated in the former rather than the latter settings. The 
conclusion of radical social evaluation authors that the scientific 
method is inapplicable to their research settings in unwarranted: 
experimental tactics are. Qften degraded by the presence of complex 
system constraints, but other standard investigatory tactics may be 
reliably applied toward replicable knowledge. Any segregation of 
social evaluation from social psychology is dysfunctional, since a 
relevant social psychological science can be approached only with the 
development of overarching theory capable of explaining interactive 
behavior in both settings. Correspondingly, social psychological 
relevance varies neither with a researcher •s investigatory strategy 
lor problem setting, but directly with the goodness of evolved 
theory. (Author) 
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SOCIAI. PSYCHOLOGY AND SOCIAL HVALUATjON 

{M}cn oij^',ht to kjiow that in the theatre of hiimait affairs it is only ^ 
for iloils arul an^»els to i)C spectators. f*rauc.is Bacon 

Generally, evaluation research may be defined as tlie conduct of social » 
scientific inquiries, usually in tl?io context of some .institution, corpora- 
tion (jr ai^^cnuy, wi\ere the i nves't i j>atory purpose is a functional assessment 
of some unit subsystem. Social evaluation, public (i.e., tax-based) sector 
evaluation, co rrespond i / focuses on some pro^^ram (0. Cook, 1966; Cirobman, 
1970; Moores, 1973; Ta/lor, 1973), policy (llyans, 1972; Weiss, 1972, 1973; 
Wozniak, 1973) or service provided by a "social" agency (DuBo i.s Ti Mayo, 
1970); the functional assessment sought is an index addressing the effi-. 
ciency or practicality of service delivery 6v program operation. In both 
cases, "tiie essence of evaluation is attribution" (Hvans, 1972, p. 634), 
where attribution is understood as a trained observer's scientifically 
guided judgment about program worth (cf. Scriven, 1967). 

Due to a sharpening accountability focus in the public sector, the last 
5 years liave seen an iiicroasing demand (cf. the APA Mon i tor) for social 
scientists competent to provide empirical indices of pi-ogram -worth.. 
Correspondingly, and prodded by the dual goad of an over saturated academic 
market as well as .strong demands for research relevance (Silverman, 1971), 
an increasing number of social psychologists are entering the social evalu- 
ation arena either as full-time practitioners or academically-based "social 
relevance" researchers. The net effect has been the quite recent appearance 
of a s i ;',n i f icant {i'.j\.y the 1973, 197^1, APA^ Pj;ograms) ])0(]y of literature 
treating the "proper" conthict of social evaluation endeavors." 

It has currcijtly ))Or()f(i(' fa ^;h i onai) 1 e in the social evaluation literature 
to empiuisi;:e the unujucncss of tliesc \u\dertak i ngs as compared to traditional"^ 
social psych{)luy»ica 1 i )n(u i i-y . ' Si nic t ura 1 1 y , al least Arj'.yris (1970), 



Hornstcin et al « (1971) and Krausc Howard (in press) have coined neolog- 
isms (e.g., "social i ntLM-vcnt ion*') desij»ned to encompass social evaluation 
concerns as well as delimit them from the remainder of social psychology.. 
Specialized journals, and even department titles complement this differen- 
tiation. . ^ 

The functional differentiation is not limited to mere nomological 

exercises, liowever. Ratlier, when one invents a new name for something, 

one must then take special pains to demonstrate the true uniqueness af 

the creation. Just such a functional segregation of social evaluation 

from social psychology has been attempted by a number of recent authors. 

These assert that the problems and settings of social evaluation are by 

nature different from traditional [mrsuits (e.g., Guttentag, 1973; Koen, 

1973; Krause P, Howard, in press ; Sechrest , 1973). Consequently, it is 

claimed that the adequate performance of social evaluation requires a new 

specialist (Krause Ti Howard, in press) versed in noVel methodological and 

conceptual tools "not likely to be found coexisting with very many social 

scientists today" (Sechrest, 1973; p. 2; cf. Guttentag, 1973; Proshansky, 

A 

1974). 

This author has no quarrel with those who would separate social evalua- 
tion, from social psychology, beyond a degree of dismay and a portion of fear, 
"i'he dismay is recurrent, and generated by any new attempt to shatter our 
already fractionated discipline still further. The fear, though, is caused 
by a recognition ol" pendulum-swing oscillations between proponent s of either 
"side" of the sort whicli Hull once characterized in another context as "meta- 
physical and theological conl i*o\/ersy" (PJ3S, p. ^192). It is true tiiat those 
with evaluation concerns historically have been adjudged as less "mainstream** 
than basic researchers (cf. Marx Hi Mix, i9()3; Samuels, 1973); it is equally 
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true that, primari]/ through the efforts of social evaluation pioneers 



(e.g.. Ar^yris. 1970; I.ewln, 1951; Scrivun, 1967) 



1 1 



. . . *applier' concerns* 



...arc no longer seen as the sordid options of mental cripples*' (Koch, 




1971, p. G72). However, just such a history makes the hypothesis plausible 




thai curreriL sugrci^a lory attempts may rcfiresent less dc natura d i fferences 




in problem settin^',s than the zealous slogans of a new and rising sect. We 
initially examine the .'ille^^ed differences between the settings and conduct 



of social cvaluatioi^ and traditional social psychological research to eva- 



luate this hy[)othesis. 



Some Tests of Parallelism 



The literature yields four major issues which purport to demonstrate 
substantive tliffcrcnces lielween the settings, problems and conduct of social 
evaluation and traditional social psychological researches.. In overviewing -s 
these, .we attempt to apply some common-sense tests of homomorphism. If 
social evaluation writer's are correct in their charges of noncomparability 
with traditional excursions, their pursuits will be seen to involve different 
(unicjue) sorts of settings, and encounter different (noncomparabl e) kinds of 
difficulties. If, however, the issues cited by evaluation specialists appear * 
to occur also in the traditional domain, even if with differential absolute 
frequencies or cmpf^asis, we may conclude that the two efforts are homomorphic ; 
(i.e., pattern-matched; they "map") and thus not substant ively differenl. 
yhc need for "new specialists" and "novel methods" would thereby become some-, 
what less clear, though a demonstration of homomorphism would not nullify 
charjics that exist inj\ methods a r'O i nsu fl'i c i cnt to both research domains. 

The I'irst issue raised cites the attributive and advocative nature of 
social evaluation pursuiti; as t^HiipartMl to tradilion:il ont^s (cf. David, 
19711. It is claiiiiod that evaluations are spec i fi ca 1 1 >• ilesi)',ned to opera t ional I ze 
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the articulated values and goals of a program, and are therefore explicitly 
conducted in order to j udgrnenta 1 1 y determine to what extent these values 
have been satisfied. Since traditional research is said to aim toward 
being "objective*' and "value-free/' social evaluation should be regarded 
as a conceptually distinct endeavor from traditional research (e.g., 
Charlesworth, 1973; Giittentag, 1973; Koen, 1973; Krause P, Howard, in press; 
Weiss, 1973). 

'I'his is 4i s i iniu taucousl y^.di f f icu 1 1 yet tenuous issue since (a) it dis- 
appears at all but the polar cases of the two settings, and since (b) earlier 
evaluation writers have .amply demonstrated their ability to carry out such 
'*attribut ivc" research within the traditional mold fcf. Scriven, 1967). 
Yet, it ought to be [)Ointed out once more that traditional research is never 
value-free or objective ("e.g., Nagel, 1961; Schlenker, 1974). If weWe 
learned anything from the social psycriology of the psychological experiment 
(e.g., harber l\ Silver*, 1 ) , it is that we must opt to make our value 
biases explicit in research or else suffer the consequences. Secondly, the 
foci and purposes of trriditional research efforts parallel almost exactly 
the description of social evaluation efforts given above. In fact, the 
Hiure frequently employed varieties of experimentation (e.g., liypothet ico- 
deductivcj have uS their formal aim the ''evaluation'* of some model of. real 
world phenomena {Ka])lan, 1964J . Finally, in both traditio^ial and social 
evaluation pursuits, the cycle of incjuiry is the same: deduction of hypo- 
theses (from a ilicory or a program), lest, and sul>sc-(|utM)l modification (of 
the theory or the pr-ogram) Botli are attributive, l)oth are value-bound, and 
botli employ the slandaiul "iii(|uiry cycle" (Marx r, Hi Mix, 1963) in pursuit of 
ktiowi edj'.e . i)n ibis "J udjMncnt a 1 a t l f i but i on'' issue, social psychology and 
social evaluation aj^pear to be higblv homomorphic. 
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The second arp.umcnt contends that rhere exists u multiplicity of con- • 
corned parties in social evaluation settings witli vital interests in the 
desip.n, collection and disposition of data relevant to any program or proj- 
ect. These parties, because of formal or informal role positions, ordinarily 
possess conflicting values and preferences uhout any researcli endeavor, and 
will attempt influence on the researcher with respect to the focus, design 
and conduct of the evaluation (Argyris, 1970; livans, 1972; esp. Krause 5 
Howard, in press; Roston, 1973; Taylor, 1973; Weiss, 1973), Consequently, 
a grand conflict of research interests may ensue, with the researcher either 
ineffectively caught in the middle or else enlisted as a partisan for some 
factional cause. It is claimed that the common existence *)f such conflicts 
renders the traditional research model emphasizing dispassionate objectivity 
and especially total experimenter control incapable of implementation 
(Guttentag, 1975; Krause Howard, in press; Weiss, 1973). 

Hven IF we grant the premises, it is not clear that the existence of 
meta-conf 1 icts in social evaluation settings renders tiicse distinct from 
traditional social j)sychological settings. Though few traditionalists admit 
it, the univcrf.i ty ar, a r.etting for basic research is n 1 so quite well des- 
cribed by the meta-conf 1 let paradigm (cf. Wolfe, 1971). The assistant pro- 
fessor at a large universit)^ like his evaluator counterpart, must also 
contend with competing research factions, whether from the chairman' s dis- 
couragement of "nnfundablc projects,'* students' reluctiuicc to play captive 
guinea j)ig, or just the pi-cfercnces ol" a journal cditoi* who regards tiic 
i nvest i gutor ' s intcM-est area as passe. Tiii s is not to say tiiat citlier tradi- 
tional social psychological resoarcli or social evaluation studies are vacuous 
proclainat ioru; "bought" by the most potent power broker in either setting. 
RathtM', wt* inst rocoi',ni/(* i ho ri'a I i ( v that I'eiuM rcbers i n un i V(M*j; i- ty labs as 
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well as^ those evaluating' ongoing social programs must contend with a number 
of parties who want, and who will exert pressure to obtain, different pro- 
cesses and products from the researcher. Traditional or evaluational, 
research is always to some degree a negotiated compromise. This distasteful 
aspect of "doing science*' has persisted from the time of the Greek scien- 
tists and their patrons to the present. It offers no substantive differen- 
tiating criterion by which social evaluation may be segregated from tradi- 
tional rosoaTTh entb'.ivors (rf. also Taylor, 197:^; Weiss, 197.*^). 

The third, political sensitivity, argument asserts that social evalua- 
tion researches are always carried out within, and affected by, the pol i tlcal 
or bureaucratic system in which the target program is embedded. Consequently, 
social evaluation efforts often fall victim to system sensitivities having 
no direct relevance to the research project, but "which may impede investiga- 
tory efforts (e.g., David, 1971 Kvans, 1972; Taylor, 1973; Weiss, 1973). 
For example, research designs potentially yielding information which would 
reflect un fa voral) I y on [>roj»ram admi ni strat;ors and sponsors may be rejected, 
or unflattering findings may be suppressed. P.ven the decision of which 
program to evaluate is political (Weiss, 1973), since more successful programs 
a)-o more likely to be suhjccted to scrutiny because of pressure from public 
officials needing favorable political ammunition. These ^'secondary concerns/' 
because of their biasing effects toward administrative protection and re- 
search censuroship, are said to render the traditional model inapplicable, 

Aj'.a i II , howevfi", i1km-(^ appears to ho no lack of j'ootl honiomorph i sm between 
the social evaluation arena and traditional social psychological settings. 
Anyone who has horn i\ nunlenurrd \):irl of an academic psychology de|)artmcnt 
is well awarr of the particularly political nature of his continuing ai)poiiit- 
mont (Wolfe, 1971).'^ Structurally, i he size and composition (e.g., sex 
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ratios) of the* department, and even the secretarial assistance provided are 
all political decisions wliich are "irrelevant to," hut impact on, the re- 
search endeavor. Functionally, some problem areas are considered highly 
threatening by administrators (e.g., race or pornography research), and may 
he d i scourai^ed . And, strorjp, Ic^/i timat ion effects exist as well, as when a 
prominent investigator opens a new area and determines for others a '*worth- 
while" problem. It is unpleasant and not very tactful to raise these points 
about traditional research. . Unless tl^ey are at least broached, however, we 
are in danger of further fractionating tl^e discipline just to maintain a set 
of convenient fictions. 

The final "cardinal difference" ])0sed is the instability argument. It 
is said 1 h;a social rv;ihi;iti()n rcsc\ircli is performed on I'jrograms or projects 
which arc inlierently unstable since tliey are designed to be both adaptive 
and evolutionary. That is, services and programs are client-centered--they 
exist to deliver service, and hence innovate, adapt, change, and mutate 
constantly in order to meet that aim. Thus, the traditional emphasis on 
constancy 'of measurable phenomena and the establishment of controllable 
treatments is simply not applicable to social evaluation pursuits, since 
L-.';:.L- dis]:)luy temporal and system instabilities rendering them immune to 
the sorts of rcscarcli forays usually launched by traditionalists (Hvans, 
1972; Guttentag, 197.'^; Krause § Howard, in press; Taylor, 197-3). 

Clearly, one could appeal to the field research tradi tion" iti classical 
social psychology to establisli homomorphism between the adaptive systems and 
investigatory difficulties of evaluational and traditional endeavors (cf., 
e.g., Webb ct al., i9()()J. However, similarities can be found using very 
"basic" examples as well. Those readers familiar with the study of choice 
behnvioi- via some I'ormal econometric i.iodel will recognize all tlie instabilities 
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claimed as province by social evaluation writers inherent in this ''lab" 
research :*.ct*tinj;. (!{)in: era i n j.^ slriiclural factors, no oao is t{\nto rlear 
what utility or probability functions look like. Regarding adaptation and 
evolution, human decision makers must be viewed as "functional gain- 
max i inizerij , " i nfon.iar. i on-processers that le.irn (change") from experience 
fe.g., Bonoma, in press) and show a shocking lack of concern for control^a- 
bility and stability as \*ell. 'i'hc essential point is just that humans, 
quite ]>leasantly, arc unstable (i.e., adaptive) systems of the most refined 
order. Regardless of the jH>int or level of application, any investigatory 
effort which deals with their behavior:; must necessarily and simultaneously 
encounter system change parameters. These parameters may be more complex 
and confounded in evaluat ion than in (say) the study oi' choice beliavior, 
but are nonetheless clearly "pattern-matched" (i.e., homomorphic) whether 
it is the adaptive system of a social agency or tliat of a single human being 
which is observed. 

Hxpcrime nt at i on , The Sclent if ic Method, and ^I'he bogie of Inquiry 
Because of these alleged differences, writers on social evaluation 
( cf. esp. Cluttentag, li)7!5; Krause U Howard, In press) contend that the 
L.Miai strategy of inquiry in social psychology, that of the experiment, is 
.unsuitable to evaluation pursuits. As Guttentag describes it, the social 
c:valuation researcher's hypotheses must be translated into^an impossible 
null testing format, preposterous assumptions of randomness made, meaning- 
less tc!.l!> u\' i j',n I i' i cance eiuploycd, and I lie entire efCorl rendei'cd futile 
by squeening it into "a classical experimental straight- jacket" (p. A), 
Krau.se ami Howard a rj',ue I he same point in a more detailed maimer. They 
claim that (a) it is impossible to select a set of independent varial)les 
wliieh produce replicable el't'ccts in a service program; (b) it is imjn>ssihle 
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to demonstrate the construct validity of any set of variable operationa- 
lizalions; and fcj ll.c: crxnplete variable set describing! a progr-.m must 
necessarily be unknown. . . 

lifiiiirary to tiiese conclusions, tiie comparisons sugj;osted alcove indicate 
that there quite j)o.,sii>ly exists a fai r-to-iiood depree oi' homoinorphi sm 
bctv/een the nature and context of both traditional social psychological and 
social evaluation vciiL^:irch settings. However, our counter-arguments cmpha- 
sizing the simi J ari t io.s -ather than differences between social evaluation 
and traditional social psychological research in no way weaken the evaluation, 
researchers* contention that the existence of conflicts, sensitivities and 
instabilities degrades the applicability of the experimental method to these 
settings. RMthcr, t liey may be correct in this hypothosin, but for tl\e wrong 
reasons. Since social evaluation settings appear to be homomorphic to tra- 
ditional concerns, their contention that system conditions often make the 
CMpcrimcnral method inc;i[>abJc of implementiit ion opens this Pandora's box for 
an_ social psychology to a greater or lesser degree. Our comments here, 
then, are directed toward both settings. 

We do not enter this controversy at its most general level, because 
scvi-iMl recent ;in<l excellent pieces exist by both supporters (e.g., Schlenker, 
1974) and detractors (Gergen, 1973, in press; cf. also Koch, 1971) of the 
experimcnlai met hod ■ and \hu in(|uiry process as it is current^ly practiced in 
social psychology. If one accepts the initial assumption that social events 
are at least partially orderly (i.e., causally produced), then it can l)e 
demonstrjiteil (cf. Kaplan, 19(».n t luU such phenomena can be most efficiently 
studied i>y way ot* the experiment, given certain intial conditions. The 
exi';lencp oi' (he t h<» appropriate "initial ronrlitions" is exnctly wliat writers 
such as liergen II97S), Kocli |19V'l) an.i Newell \\97:>) question, and contrasting 
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that the existence of uoiulitions such ;is those cited by social evaluation 
special h,s often renders cxper imcnt a V i on , witli i ts rec{u i rement of rigorous 
control, imj)ractical or impossible. Moreover, this is oft^n the case in 
social evaluation as well as traditional social psychol o|i;i ca 1 research set- 
tings. The qualifications, and our disagreement with segregationists (and 
to some extent, with those v;ho view social psychology as history), arc that: 
(^1) these critics often choose to focus on only the simj^lor, and hence most 
degradablu, uxptM* i mem a 1 me i hods in their attacks; (2) I hey adopt a rather 
parochial view of what, consti tutcs the scientific method generally; and (3) 
socinl evaluation authors in particular seem to confuse the scientific 
n.vLhod with the logic of scientific inquiry in psychology. 

Concoi-ning the i'irst point, it is safe to say that the scientific method 
(e.g., Kajiian, 19(>/l;"Marx l] Hillix, 1963) includes more options under the 
heading "experiment" than the typical garden variety factorial design. 
Sophist icaled and ion orthoj'onal desi}»ns, many incorporating intentionally 
coni'ounded factors and temporal variations, exist 'wincii are more suited for 
i nvc'U i )'a t i n j; complrx ;r. oppoi.i'd to stmplr phcnoiiiri};i (cf. (!aiiipj)cll l\ 
Stanley, IDG^; Winer, MJ71). Statistical assumptions and data analytic 
procedures are ordinai'ily robust enou ',h to allow l)oVh ad hoc treatment 
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their views with the arguments of social evaluation researchers shows the • 
latter arguments to be specific forms of the general questions raised by 
the former, l-or our purposes, the question translates as, "Does the 
existence of meta-confl icts of interest, political sensitivities and a focus / 
on unstable phenomena degrade the appl icabi xity of experimental investiga- 
tory strategics in traditional as well as evaluation applications?" 

Contrary to Schlenr.cr fl974), we believe the answer must be a qualified 
"yes." That is, and in partial agreement with Gergen and Koch, it is true ,) 
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assignments and the collection of only the most raw sorts of ordinal measures 
with little loss of inference ai)jlity fcf. Games Klare, 1968'^. When even 
such "reU»xod" experiments cannot he attempted due to system constraints, 
cxpcr i niental tactics include as full-fledged members certain uses of the 
simulation (cf. Abelson, 1U08) and other operating representations of com- 
plex systems. And, when the system approached is only marginally understood 

in its full complexity, the more advanced correlational (e.g,, covariance, 
♦ 

path ;inaiysjs, t ini(» - l;i);)'.c»(l analysis] dcsij>ns and data i>a i t i on 1 ng techniques 
allow teasing out experimental effects from system noise. Thes::? experimental 
tecliniqucs are available to our current specialists, both in the traditional 
and social evaluation areas, and require neither new researchers nor novel 
metliods for their implementation. 

Tt remains true, however, that even the more sophisticated experimental 
designs may be basically incompatible with the system constraints existing 
in social evaluation settings. This is because of the ruiture of the experi- 
ment, which is designed to serve as a '^snapshot" of effects produced under 
specified system states. Taking such a picture is of little value if the 
subject clianges immediately after exposure, and of very little value at all 
if raicb changes are tlu; result of systematic differences in powerful factors 
assigned to "error- variance" (e.g., political sensitivities) completely 
C)Ut?/ide the realm of expe r i incut a I i nt erest . Therefore, the usual strategy 
. of inquiry, winch i nc 1 udcs formi ng a rudimentary "map" (i.e., theory) of 
the invesl i )',al()ry area and I hen exp 1 u r i ng this map via exper i meiil at i on , may 
he dop.radcd for the reason that the entire map is nonraiulomly affected by 
its enc 1 osu re wa Ml i n [)r()gressi v c ly larger systems. 

It i'. nut (iir (•xi'MriuH' of !',ystem:; within systctir; per so which dej»rades 
the apjilicalion oi' exper i moiual i on ; as Shoostcr (PJ71) has ]Knnted out, even 
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extremely complex systems are amenable to classification and, experimentation. 
Hathcr, the basic problem is>to specify an appropriate level o^ analysis for ' 
*'map** formation befo re ex])eriments are undertaken: it makes little sense to 
atceiiipt theory-formation about (say) small group interactions within a ser- 
vice 01" pi'oyram when ilicse arc known to be * nonrandomly affected by the sort 
of macrosystem constraints addressed by social evaluation methodologists 
(cf. also j.uszlo et al . , 1^74). Unfortunately, detailed knowledge of system 
coni'ounds and constraints 1 5; ordinarily not possessed for any specific set- 
ting prior. to invest igat ion--such knowledge only accrues through repeated 
unsuccessful investigatory attempts. Social evaluation researchers, then, 
directly encounter the "dilemma of complex systems" in their avocation: a 
sufficiently articulated theoretical "map" of the investigatory area is 
needed to permit hij>,hly controlled observations, but good experimental 
observations, must avvait a detailed map which distinguishes exprimental 
effect from system noise. *I'his dilemma may be presumed to have led both 
to social evaluation's alienation i'roin the experimental method and from 
traditional social psychohigy. 

If the cui'rent arguments regarding the homomorphism of evaluation and 
Iradltional social psychology are valid, we should expect that traditional 
endeavors encounter a similar form of the dilemiiia of complex systems. This, 
appears to be the case. With regard to dissonance phenomena , for example, 
it j^emains {after 17 yoarsj impossible to specify complete variable sets or 
evf.Mi h i j>li I y prohal) 1 c fuiKMional relations in current explanatory efforts 
(cf. Kehiian, \97<\] Krause, 1972), a phenomenon which parallels the Krause- 
lloward complaints ab'out evaluation endeavors. Kather I'etardecl progress has 
jituMi larj'c-ly observable nut only I'oi- dissonrnicc hut for most traditional 
social psychology e,)\., Koch, 1)71') primarily btu-auso the dyad and the 
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small group have only recently been taken seriously us ( I'J social systems 
whic-h (2) arc s i jmi i f i cant 1 y alTc'ctcd by tbo ot her systems in whicli they arc 
cniboddecl. l-or the first, we now talk about experimenter-subject interactions, 
when we very recently used to conceive of our discipline as the study of 
individual respons/»s to social stimuli (Shaw Costanzo, 1970: p'. 3). With 
ref^ard to the second, we currently seek knowledge which goes beyond the 
college soi>iiomore population in its generality, realizing that the various 
macrosystems in which subjects and observations arc embedded is an integral 
part of "objective" data production (cf. Miller et al., in prcss; . This ^ 
change in thinking has been no less than revolutionary for traditional 
social psychology, and has led to the dual result of producing more genera- 
lizable data while simultaneously awakening traditionalists to the awareness 
that they must confront system dilemmas as well. The dilemma of complex sys- 
tems has been less visible in traditional social psychology just because 
such researcli attempts to minimize cross-system influences (i.e., to maximize 
control) in maximally simple social systems (e.g., dyads). That such mini- 
mization could not (and should not have been expected to) eliminate all 
system influences is a phenomenon with which we are just now learning to 
deal (e.g., Kelman, 1974),. 

Thus, simple expo* i incutat ion in both the social evaluation and tradi- 
tional social [)syehological domains may be degraded by system constraints 
impinging on the investigatory area. This "dilemma of complex systems" is 
ol'icn mure po i j'.na n i 1 y ix p«- r i i-ntcti by r.oci;il c\/a!ualion rer.carchers than by 
ti-aditional social psychologists because (1) the former do not (and cannot) 
alU-mpi lo minliiii/c' cioi.-. sysli-m inriuences, and (J) I lie [alter ordinarily 
restrict themselves to the least complex, and hence most controllable, sys- 
tems, in r:iirnt^ss to ;;ocial evaluati)n authors, we must agree that greater 
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degradation of the experimental method often is experienced in program 
evaluation than in xho study of dyadic behnvJor. lIov;ever, the ^o// level 
and power of traditional social psychological laws [cf. Gergen, 1973) at- 
teh;ts at least partially to the existence of a homomorphic dilemma within 
the traditional doniain ar> well. Thus, tlie settings and problems of the 
one cannot logically be segregated from those of the other on these 
grounds. It can l)u concluded that no investigatory-explanatory approach 
whicli ii'.noros the mixw-, of systems in which the phenomenon of interest is 
embedded can produce ,gcneralizable knowledge. Fuirther, it may be expected 
that both traditional arui uvaluational exper imentri 1 pursuits will gain in 
feasibility with (1) the methodological sophistication of the researcher, 
.ind (2) the a I't i cii 1 a I i f>n dI' ycncvul j) r t nc i {) I cs of c ros s - system influence 
(cf., e.g., Grinker, 1967). 

While a systems theory a[)]n-oacli to interactive events may eventually 
offer better "maps" whicii will increase experimental applicability and 
results general izal)i Ij ty, utlier tactics subsumed in the scientific method 
offer mor-e immediate ai:>iilication. Most prominent among the usually ignored 
observation techniques detailed in every introductory text is that of 
ii^i ur.d isl ic obsei-vat ion . As Ciiarlesworth (197."^) points out, the difference 
i)etwccn exi)er inien tal studies and naturalistic observation is just that the. 
fornjr^)- conconli-ate on wlirit an oi-j'.anism (or, organ i ?.at i on ) can do under 
specified and known system conditions, while the latter conceiilH^a-t-e-S" on-- — 
what '.\n or);:iin!;m docs tlo uiulcr {ho operative system constraints in its 
environment (even i 1" th(»se ai'C cui-i'cnlly unknown). Naturalistic observation, 
add i t i o/ia ! 1 y ofl'crs advanlaj'.cs beyond u simple iiiel hod of observation. '\'hc 
WfM-k of a number of !;eri(>ns students of social cthoio)»y (e.g.. Bales, 1971; 
Barker, lOf^!^; Ciia r 1 eswo , 197!^) suj>i'.csts that tlie raw behavior of humans 
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in their social habitats often divides itself into chcorctical ly useful 
cate^U)ries and typCilc^jMcs (cj;., Barker's behavior settings), Thorefore, 
such studies covild serve the dual purpose in both social evaluation and 
traditional social psychology not only of establishing a raw data base of 
ongoing h';haviur, but also of providing strs)ng pre-cxper iinental indicators, 
of which system facets and behavioral regulaauties are of crucial theore- 
tical concern. 

Again, the caso history method, while oidinarily not seriously consid- 
ered by traditionalists, may serve as an extremely valuable tool in both 
traditional and evaluation studies by wliich to circumvent the dilemma of 
complex systems. The collection o'f a wide variety of historical and current 
reports by members of a service unit (or parties to social interaction) may 
bear the im])rinteur of secondary data as opposed to more '^behavioral" mea- 
sures. Case studies have, however, the advantage of permitting economical 
comparisons l)etwcen the numerous compotinj', factions encquntered in complex 
systems, as well as quickly pointing out "deviant cases" among factions .for 
special tlicoretical or subsccjuent experimental focus. When combined with 
naturalistic observation of a service* unit , such techniques allow at least 
ifjti;;h convurijcni valitlily estimates to 1)0 performed on otiierwise uiimeasura.bl e 
system or behavioral components (cf. Uapoport , 1968). 

Thus, the second major point is this: writers on social evaluation, 
as well as some of tliose critical of traditional . social psychology, have 
taken a ratlier parochial viewpoint by e(|uat ing certain restricted aspects • 
of the experimental method witli tlic scientific method, .Then, l>ecause it 
can be argued that a[)[)l i caj ion oF the former i s often degraded by conditions 
i\n\m\ to f^visi in i-omplex sy.tc^ms, it is cotU'luded lliat novel methodological 
techniques :i!ul new specialists are di.iuinded. However , otlier "standard" 
I I'f 1 1 o r y I <•( liii i qiM- , tn \rii'ni<' < rr av.TlliibIc whirh arc not a:; prone 
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to system degradation as the experiment, for they do not require the same 
stringent control specifications and are more suited to exploration in the 
absence of a well-arCicuUited theoretical "map." The segregatory conclu- 
sion of the more extreme social evaluation authors is unwarranted; rather, " 
it is only necessary to look beyond the n-group factorial paradigms on 
which we are functionally fixated to the remainder of the techniques com- 
prising the scientific r.iethod. Those who criticize transplantation of 
traditional simple cxper i mont at ion to complex evaluation settings are 
correct when they assert chat these are ordinarily degraded there, but 
incorrect wherj I hey conclude that new methods arc urgently needed or a dif- 
ferent subdiscipline thereby comes into existence. Conversely, those who 
always choose their basic research problems to fit the already-prepared 
bed of orthogonal variance estimates are equally incorrect, since they 
fail to perceive that traditional social psychological research is in 
principle no less a social influence setting, no less political, no less 
unstable than a confounded service agency. 

Concerning the final ])oint, it appears that social evaluation writers 
have chosen to equate the scientific method with what is ordinarily known 
a:> iho strategy of in(|uiry iii science (e.g., Ilomans, 1967; Marx P, Hillix, 
19G3). That is, the inquiry process is usually thought of as having two 
.stagtrs-: -^discovcry and explanation.' Those tactics included in the scienti- 
fic method (experimentation, naturalistic observation, etc.) facilitate the 
discovery of "facts" in a manner C<-'-g-, repl icabi I i ty , public verif iabi lity) 
consistent with s^Heruiric cMuk\*Jvars, while tl\c meelianisms of law, model and 
theory allow .subsuming such t'acts under more general explanatory principles. 
J* i '-^ thn'.r pr i nr i p h-s , when convtM-j't^nt 1 y confiniUHl by diTferent ojieration- 
alizations and across S)-sC(.Mn levels, which arc cxiH'cictl to be genera 1 i zabl c 
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to thcorctj cally similar phenomena. To the extent to whicii social evalua- 
tion researchers have attempted to apply exporj mentiit ion directly to , public 
sector phenomena, neglecting the prior steps of hypothesis derivation and 
the succec-Jing steps of theory revision, they short-cut the scientific 
method. And, whenever the logic of inquiry is short-cut either by design • • 
or system constraints, nongeneral izable knowledge is the result. Experi- 
mental results per sc w.;.-c never intended to be general izable to even 
moderately similar phe/iomona (cf., e.g., Marx f, Hillix, 1963). Rather, 
these discovery tools provide one of a number of ways to accrue facts in 
the service of theoretical (i.e., explanatory) notions. These theoretical 
propositions, and only these, are expected to be generalizcble to between- 
system phenomena other than the one currently being investigated. At base, 
then, those who would claim the scientific method is inapplicable to complex 
social settings are f j ust i f iably) lamenting the lack of theoretical princi- 
ples to guide them in their study, and confirming first-hand that the 
discovery tools of science cannot be applied in the absence of the explanatory. 

In sum, the present view argues that the settings and problems of tra- 
ditjonal and social evaluation research are homomorphic, and consequently 
haw similar (but not completely equivalent) consequences regarding the 
applicability of oxjior i men t a t i on in particular and the scientific method in 
general. Hxper i meat at i on often cannot be easily applied to social evaluation 
problems because of the 'Mi lemma of complex systems:- however, a less- 
rc^cognizeil imt hoiiioiiu) rjih i c il i U'imm. e.xists fo r. t rad i t i oi.a 1 research as well.. 
Fortunately, other standard discovery methods exist less subject to .system 
dcgradatioi.. Regardless of di.scovery techni(,uo a|>plied. though, collected 
data must he incorporated within viable theory in order to be general izable. 
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Inquiry and Relevance 
Art iculatioii of the lioinomorph i c but differentially intense dilemma of 
complex systen^s as it affects traditional social psychological studies and 
social evaluation pursu.its offers a clarifying and integrative set of sug- 
gestions ,(i\\:irdir\g the patlt of researcli in jach setting. Wlien combined 
witli the stereotypic motivational biases of traditionalists versus social * . 
evaluation researchers (e.g., I-vans, 1972; Meehl , .1972) , it offers as well 
some commentary regjiPdinc; tlic achievement of "relevance" in either endeavor. 
We turn to such considerations by way of conclusion. 

In the present v i cw, arguments about the segregation of socia-1- evalua- • 
tion from traditional pursuits, alleged motivational deficits in one or 
anotiicr camp, and differential claims (:\nd disclaimers) of relevance all 
stem from an impropeT understanding of the strategy of scientific inquiry 
as it applies to human social behavior. Put in oversimple fashion, re- 
ivcarchers employ tilie scientific method in order to add support^ or to aid 
in modifying t hef.)ries of social i:)ehavior. No one research application is 
^'ii^^XiLDt- ^^^^ sense of.beini; directly applicable to the real world, but 
a "net" ol" empirically confirmed and extended tiieoretical propositions jiuiy 
be :uibscqucnt ly and rightly employed to generate a coherent set of decision 
rules. Thusc rules, if generated from established and cross-validated 
tlicory, will carry enough external validity to focus observational strategy, 
suggest variable sets, and so on wlien similar but more complex (i.e,, 
c ross -sy ;>lem ) phenoniciKi arc addi'esscd. 

However, the construction of sound- theory in either setting may be 
approaclicd in soiMcwhat diflerent fashion due to differences in tlie intensity 
of the dilemma of complex systems. Social evaluation researchers, which 
deal with maximally complex piuMiomen i embedded in I'ecursive and reflexive • 
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systems, will probably initially find naturalistic observation and the case 
method initiuliy less dci^radnhl c than experimentation per se. Traditional 
social psycholo)^/, with its focus on simple systems ordinari ly taken ^'out of 
context," can easily continue to apply experimentation techniques as long as 
there is :lu incrcasinji rcco^',n i t ion that system interact.ioiis cannot be »'con- 
trolled out" of observat ion settings. Thus, social evaluation and social 
psychology may initiate research projects in somewhat ojjposite manners for 
"best" theoretical progress: the former mi^h;: start with naturalistic ob- 
servations and case studies to delineate the rougli system boundaries, and 
projtress to expcr imeni at i on only as cross-system inriuences ])ecome more 
clear. The latter might initiate rigorous experimentation as the most ef- 
ficient m'-*thod of answei-injj causal cjuestlons in controlled microsystems, but 
must relax its methods periodically and convergent ly to determine if the 
questions asked have any relation to those occurring in the nexus of systems 
in which the phenomenon is ordinarily embedded. 

it is clear that both traditional social psychology and social evalua- 
tion research are impaled on different horns of the same dilemma of complex 
systems. That their problem settings force opposite sides and different 
u.-;. i i"'.' a.-it ions of the dilemma should not blind cither lo the fact that (1) 
this in no substantive way distinguishes the efforts of the one from the 
other; (2) the "nonpreferred" side of the dilemma must eventually be addressed; 
arui (3) no artificial segregation of evaluation from traditional pursuits 
can lead lo i he rauccssful r'esolulion o C tlie dilemma ol" complex systems. 
Whether one's initial preferences lie with articulated maps of systems so 
simple and conliollcd lhal the resultant rigorou.s knowledge is not very 
general ir.ahle, or with complex real woiid endeavors yielding only the most 
confounded inLiiccs in ihc al)!>ence oC i good "mai)," only the convergent 
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pursuit of both approaches can ever result in the understainlinj» aad predic- 
tion of interactive events. It follows conclusively that to sf*yiregate 
"so^cial evaluation from social psychology is not only illogical because of 
their hiy.h degree of hoinomorphism, hut eventually will be fatal for both 
as we 1 1 . 

Regarding differential claims to relevance, it is clear that relevance 

resides not in the observational field, but in the resultant theory. 

■ ♦ 

Neither traditionalists nor social evaluation researchers have reason to 
denigrate the activities of the other, since we have seen both are connected 
by a common problem. 'I'o tlie extent to which soc ial" cva hiation researchers 
are Ttcfeorypically represented as more interested in immediate applications 
of knpwlcdgc than traditional social*psychologists , these liave historically 
felt justified to claiming '*face** relevance (i.e., fir^Jt-order relevance: 
Mcchl, li)72). Conversely, traditionalists, because they currently possess 
the most viable of our theories- in social psychology, have felt justified 
in claiming *'eons t r'uct relevance (i.e., second-order relevance).^ Our 
])rescnt recognition that both encounter the dilemma of complex systems in 
somewhat different form ariuies strongly that neither faction should be too 
L.'j^jcr to disclaim the other's motivations, since both will eventually encounter 
the problcirriof ti^e other. Further, and if performance is to judge, neither 
faction i\as been very relevant at all. The real v/orld applications of 
evaluation specialists have more often resulted in confusion than clarity 
about piiblif sector proj'.i'ams, while the theories of traditionalists have 
often been so system-bound as to be ungeneral izabl c . Since both "factions*' 
apiiear to. liave the same u I t Imat e mot i vat ion, all)ei^t ttirough different imple- 
mentation strategies, it would seem that differential claims to relevance 
are unfounded, 
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More important, and partially connected, is that history will judge 
only those of our number as relevant who participate in the development of 
articulated tlioory about interactive* events. Articulated theory is not 
acliicvod l)y making a clioicc about wliethor to focus on microsystems for 
precise map formation, or on real world pro^jrams to garner generalizable 
data. Rather, only when the dilemma of complex systems is resolved in the 
accomplishment of both These facets in one social psychology will differ- ' 
ential relevance claiins become feasible. These will then consist of pitting 
competing theories against one another, not competing researchers. 
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Footnote 

I. fU'(inV-!.i i'ur ri'\)v\ni \ m:\y ho sent to the author at the Institute for 
Juvenile Kese;irch, Department of Mental Health, IMO South Paulina Street, 
Cliieaj'.o. Illinois, (yOUl'Z, The author wishes to express his j?.rntitude to 
Merton Krausc ?huI Kenneth linwavd for discussing their work, examining his, 
and encouraging the differences. This paper is not intended to represent 
the policy oi- views of the Illinois Department of Mental Health with regard 
to social evaluatioTi research. 
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*' i'ootnotes 
I . (on separate paj^cj . 

2. We restrict, our present comments primarily to the more recent (1972- 
1974) treatments of social evaluation research. Much'earlier literature 
cxi:.t:i, especially, from education and j^overjiment (e.g.. Cook, 1966; DuBois 
Ti Mayo, 1970; Crobiijan. 1970; Scri vcn, 1967), which treats the evaluation 
er.terprise as a direct . xtension of traditional researcli procedures. It 
is tiuis at least partially immune to the criticisms raised here. The 
curious reader might compare an early witii a more recent piece on social 
evaluation to determine for iiimself the radical clianj»os wiiich have occurred, 
3. Kraditional here has as its meaning the design and conduct of cQnIrolled 
expcr-iiiieala i i nvest i }'.a I ions , usually witiiin a lai^oratoi-y context. 
A. Wolfe (1971) iuis devastat ingly described the conduct of traditional re- 
:.earch in a univci-sity environment, and the numerous cross-cutting pressures 
impini»in}'. on the so-called "impartial" scholar, lie concludes, "We have now 
arrived at a definition of a successful scholar, lie is a person who con- 
stantly reiterates difierent aspects of the same idea in n manner determined 
for [\im by others without being critical of the conditions which shaped his 
' < ' (|>- ^'^')' While it is true that evaluation specialists are "selling" a 
research product, it is no less true of the academician (e.g., the emphasis 
on statistically "s i gn i f i CtUit" rcsu 1 ts) . 

however, (juite correct to argue that either the greater frequency 
of such problems, or the pi-esence of a greater number of such issues, renders 
tiu^ :w)eial evaluation settinjj practically if not principally distinct from 
the traditional (see belowj, 

(). Thotij'ji one should not single out Meehl here, since he just nicely stated 
a distinction recurrent in t he 1 i t e ra i u )-e since tho'^O's (e.g., Marx l\ llilli 



1963), the artificial positing of different *'sorts" of relevance is both an 
irulicator of ;i basic misunderstanding of the inquiry process and un early 
sign of the segregatory disease. 



